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- _ _ =- Abstract 

In all embedded computing systems, some delay 
exists between sensing and acting. By choosing an 
action based on sensed data, a system is essentially 
predicting that there will be no significant changes 
in the world during this delay. However, the dynamic 
and uncertain nature of the real world can make these 
predictions incorrect, and thus a system may execute 
inappropriate actions. Making systems more reac- 
tive by decreasing the gap between sensing and action 
leaves less time for predictions to err, but still provides 
no principled assurance that they will be correct. 

Using the concept of predictive sufficiency de- 
scribed in this paper, a system can prove that its 
predictions are valid, and that it will never execute 
inappropriate actions. In the context of our CIRCA 
system, we also show how predictive sufficiency al- 
lows a system to guarantee worst-case response times 
to changes in its environment. Using predictive suf- 
ficiency, CIRCA is able to build real-time reactive 
control plans which provide a sound basis for per- 
formance guarantees that are unavailable with other 
reactive systems. _ 


Introduction 

Traditional AI planning systems 3, 10,15 have been 
criticized because they may spend large amounts of 
time building a plan that is out-of-date before it can 
be used, and thus the actions that the plan chooses 
may be inappropriate. For example, consider an in- 
telligent autonomous vehicle that is waiting at a red 
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light. When the light changes to green, the vehicle's 
sensors detect the change and, after some further pro- 
cessing, the system decides to move through the inter- 
section and on to its destination. But, if the system 
spent too much time planning its entire route, the 
light may have changed back to red, and the plan's 
first action would be “inappropriate ” 

In response to this critique, researchers have de- 
veloped reactive systems 1,2,4,6, 13 that perform little 
or no lookahead planning, instead choosing actions 
based on current sensor inputs. One goal of this be- 
havior is to keep the selected actions appropriate to 
the current situation: because no planning is done, 
an action can be chosen quickly once sensor readings 
determine the current situation. 

However, because computations can only occur at 
some finite speed, there will always be some delay 
between sensing and action. During this “sense/act 
gap,” sensed information is stored in the system, ei- 
ther explicitly in memory modules or implicitly in the 
communication and processing mechanisms of the sys- 
tem. By choosing an action based on that stored in- 
formation, the system makes an implicit prediction 
that the stored information will continue to provide a 
sufficiently accurate representation of the world . 5 

Because real-world systems are dynamic and some- 
what uncertain, such predictions are inherently risky. 
Gat 5 suggested that these predictions and the asso- 
ciated stored internal state are useful only at higher 
levels of abstraction. We argue that, because the gap 
between sensing and action is inevitable, it is not the 
abstraction level but the magnitude of this delay (and 
the requisite prediction) that is critical. Systems in 
dynamic worlds must be “real-time,” in the sense that 
the utility of the system’s computations depends not 
only on their result, but on when that result is pro- 
duced . 14 To guarantee correct performance, an intel- 
ligent real-time system must ensure that the actions 
it chooses are appropriate for the actual current state 
of the world, not just the state of the world that was 
last sensed. 
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Rather than solving the real-time problem, reac- 
tive systems simply operate in a “coincidently real- 
time” manner 7 — they function as quickly as possi- 
ble, in the hopes that the sense/act gap will be re- 
duced so much that significant world changes cannot 
occur during the gap. In this paper, we present a 
more rigorous approach to dealing with the sense/act 
gap. Our approach consists of proving that signifi- 
cant world changes cannot cause a particular selected 
action to be inappropriate, by verifying that the pre- 
dictions spanning the sense/act gap are valid. 

In the next section, we lay the foundations for 
this proof by defining the “interval of predictive suf- 
ficiency,” or the time during which an observation 
provides sufficient evidence to accurately predict the 
value of some proposition. In the following section, 
we illustrate how explicit reasoning about predictive 
sufficiency can be implemented, with examples from 
CIRCA, the Cooperative Intelligent Real-time Con- 
trol Architecture. 8,9 We describe how CIRCA uses 
predictive sufficiency while building real-time reactive 
control plans, to guarantee that the system will never 
choose inappropriate actions or miss real-time reac- 
tion deadlines. This paper concludes with sections 
discussing the type of knowledge that is required for 
reasoning about predictive sufficiency, and pointing 
out future directions for this research. 

Defining Predictive Sufficiency 

To accurately describe the concept of predictive suf- 
ficiency, we must begin with some notation. We will 
use a simple temporally-qualified modal logic to de- 
scribe the state of a control system s knowledge. The 
logical statement K (p[t {] , tj) indicates that the sys- 
tem knows, at time tj , that the proposition p holds 
at time f;. For convenience, we will also use state- 
ments of the form K(p[tontp],tj)> indicating that the 
system knows, at time tj , that p holds continuously 
over the time interval from t a to tp. 

A control system's operations can be generally ex- 
pressed as the acquisition of a sensory observation, 
the logical deduction of what that observation means 
about the state of the world at the time the observa- 
tion was made, the deduction of the predictions that 
the observation allows the system to make about the 
world following the observation, and the selection of 
an action based on that knowledge. In our notation, 
we have: 

0[ti] 

\ interpret 

Vp 6 ft ■■ K(p[ti],tj) 

| predict 

V</ £ Q - K tqp]i t-k) 

j, select 

tap] 


where 0[fi] is a sensory observation made at time 
ti, ft is the set of propositions which can be inferred 
about the world at time ti from the observation, and 
Q is the set of propositions that can be predicted over 
the respective intervals f?/?]- These intervals are 
the “intervals of predictive sufficiency,” during which 
the observation O is sufficient to predict the value of 
the propositions Q. The time tj is the time by which 
the system has derived its knowledge of P , and tk is 
the time by which the system knows Q. Following 
those deductions, the action a is chosen and executed 
during the time interval [t aa ,t a p]- 

We first use the concept of predictive sufficiency 
to show how an action can be guaranteed to be ap- 
propriate when it is executed. The key to avoiding 
an inappropriate action is to ensure that the value of 
the propositions used to choose an action will remain 
unchanged long enough to keep the action appropri- 
ate. This can be achieved by making action choices 
based on propositions whose intervals of predictive 
sufficiency cover the time during which the action’s 
preconditions are necessary. More formally, suppose 
the action a requires a set of propositions R to hold 
during the respective intervals U rat tr »]. If ft C Q and 
Vr £ ft : (<rc < <r a) A {t T p > t rb ), then the intervals 
of predictive sufficiency that are supported by the ob- 
servation O ensure that the required propositions will 
hold as necessary. 

For example, in the stoplight scenario described 
earlier, the vehicle agent will at some point make an 
observation confirming the proposition “the light is 
green” (ft). This proposition alone is not sufficient 
to justify crossing the intersection, because there is 
no guarantee that, at the time tj when ft is known, 
the light is still green. The knowledge resulting di- 
rectly from interpreting sensor readings can only de- 
scribe past states of the world. However, if the system 
knows some information about the domain s dynamic 
behavior, it can derive additional propositions that 
describe the current and future worlds. In this ex- 
ample, the system might know that the traffic signal 
will switch to yellow for at least five seconds before 
it turns red. So, although the system does not know 
if the light is still green, it can conclude that, for at 
least five seconds after the light was seen to be green, 
the light must be either green or yellow, and the in- 
tersection will be “safe” to cross (Q). If the agent is 
sure that the time it takes to infer these propositions 
from its observations and cross the intersection is less 
than five seconds, it can guarantee that it will never 
be in the intersection during a red light. 

Thus the addition of domain modeling informa- 
tion has allowed the system to make explicit pre- 
dictions about the future state of the world, based 
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on stored sensor readings. Given further information 
about the agent’s own performance, these predictions 
are then shown to be sufficient to justify certain ac- 
tions. This example illustrates how predictive suffi- 
ciency can cover the sense/act gap, avoiding inappro- 
priate actions. 

Implementing Predictive Sufficiency 

In this section, we provide a high-level description 
of CIRCA and show how the prototype implementa- 
tion of the architecture explicitly reasons about pre- 
dictive sufficiency and makes guarantees about its 
behavior. Note that we do not claim this imple- 
mentation is ideal; it serves only as a useful testbed 
to demonstrate the concepts of predictive sufficiency. 
More details on CIRCA are available in related pub- 
lications. 8,9 

Figure 1 illustrates the architecture, in which an AI 
subsystem (AIS) and Scheduler cooperate to strategi- 
cally plan and schedule a set of reactive behaviors that 
will cope with a particular expected domain situation. 
The parallel real-time subsystem (RTS) is guaranteed 
to accurately execute the behavior schedules, com- 
prised of simple situation-response rules. In this pa- 
per, we are focusing on how the prototype AIS explic- 
itly reasons about the sense/act gap and predictive 
sufficiency while planning reactions. Note that this 
lookahead planning is performed while previously- 
planned reactions are already executing on the RTS, 
so the planning process can be viewed as “off-line.” 
To show how CIRCA uses predictive sufficiency, we 
must first briefly describe the system’s world model- 
ing techniques, which it uses to reason about the be- 
havior of the world and the actions that the system 
should take to achieve its goals. 

In the prototype implementation, the world model 
takes the form of a directed graph in which nodes 
represent possible states of the world and arcs rep- 
resent instantaneous transitions between states. The 
status of ongoing processes in the world is explicitly 
encoded into the representation of a state. Important 
changes in process status thus correspond to transi- 
tions between states. The model distinguishes three 
types of state changes: action transitions , performed 
deliberately by the system’s reactions; event transi- 
tions due to external world occurrences; and tempo- 
ral transitions , due to the passage of time and ongoing 
processes. Timing information is associated with each 
transition, representing constraints on how long the 
world must remain in a state until the transition may 
occur. We now illustrate how this model is used by 
the AIS to explicitly reason about the sense/act gaps 
that will occur when planned behaviors are execut- 
ing on the RTS, and how the system guarantees that 


those gaps will not lead to inappropriate actions. 
Avoiding Inappropriate Actions 

Figure 2 shows an example portion of the graph- 
based world model for the stoplight scenario described 
above. Within the state descriptions, the model 
shows that the stoplight can take on its three sig- 
nal colors, Red, Yellow, and Green. In the Yellow 
and Green states, it is safe for the agent to cross 
(“Safe2X”), but not in the Red state. In this sim- 
ple example, we have abstracted out all of the agent’s 
own state except for the indication of whether it has 
crossed the intersection or not. The different states 
of the traffic signal are connected by temporal transi- 
tions (double arrows) indicating that, as time passes, 
the signal will transition to subsequent states. Each 
temporal transition is labeled with the minimum pos- 
sible delay before the transition occurs, perhaps de- 
rived from the agent’s previous experience with this 
traffic signal. For example, the transition between 
the Red and Green states indicates that the signal 
will stay red for at least 60 seconds before turning 
green. 

When planning reactions to operate in this domain, 
CIRCA does not build an enumeration of possible 
world states and then plan actions; instead, it dy- 
namically constructs the graph model and the plan 
of actions together in a single depth-first search pro- 
cess, essentially similar to a forward-chaining STRIPS 
planner. 10 This process operates on a stack of world 
model states, examining each state in turn and plan- 
ning actions that achieve goals and preempt temporal 
transitions that lead to failure. 

To begin the planning process, the initial states are 
pushed onto the state stack. Then, as long as the 
stack is not empty, the system pops a state off the 
stack and considers it the current state . The system 
simulates all of the event transitions and temporal 
transitions that apply to the current state, yielding 
either new states that have not been examined yet or 
states that have already been processed (i.e., states 
for which actions have already been planned). New 
states are pushed onto the state stack, while old states 
are simply updated with the information that they 
have a new source state. The system then chooses an 
action to take in the current state, as determined by 
a heuristic scoring function. 

For example, if the system is told that the “red” 
state A is its initial condition, it will first consider 
the applicable event and temporal transitions, push- 
ing the new “green” state B onto the stack. The 
system will then try to plan an action for state A] 
since the state is not safe for crossing, the only ap- 
plicable action is no-op (shown as a dashed line in 
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Figure 1: Overview of CIRCA. 
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Figure 2: An abstracted portion of the world model for the stoplight scenario. 


Figure 2). The system will then mark state A as pro- 
cessed, pop state B off the stack, and derive the new 
successor state C via the temporal transition indicat- 
ing that the light will change to yellow. Again an 
action is chosen for the current state, but this time 
the cross-intersection action is chosen because it 
is applicable (Green is safe to cross) and because it 
leads to the desired result. So at this point CIRCA 
has planned a simple reaction indicating that, when 
the light is green, the agent should cross. But the 
system has not yet shown why this action is guaran- 
teed to be appropriate when executed; it has not yet 
addressed the sense/act gap, and the possibility that 
the light will change before the cross-intersection 
action is completed. 

CIRCA addresses these issues by ensuring that the 
propositions used to satisfy the action’s preconditions 
are covered by intervals of predictive sufficiency. The 
system knows the worst-case execution time of all of 
its sensing and action primitives, as well as their com- 
binations. Thus the system knows exactly how long 
it will take, in the worst case, to detect the green 
light and cross the intersection (here, three seconds). 
To check for predictive sufficiency, the system must 
look for other domain processes that may be occurring 
during the action (i.e. , transitions to other states). In 
this case, the system has recognized, based on domain 


knowledge, that there can be a temporal transition 
leading from the green state B to the yellow state C 
after a minimum of 25 seconds. 

As noted above, CIRCA does not know how long 
the light has been green when it is observed; therefore, 
in the worst case, it is assumed that the temporal 
transition to the yellow state C occurs at the same 
time the system initiates the transition to cross the 
intersection. This corresponds to the “ghost” action 
transition in the figure (the dotted line), showing that 
the action planned for state B may actually be applied 
to state C, leading to a new state £ where the signal 
is yellow, but there is now a minimum of only two 
seconds before a temporal transition leads to a red 
light state. 

In this process of looking at transitions out of the 
state for which the action is planned, CIRCA has 
shown that, although alternate results are possible, 
the precondition of the action (“safe2X”) is known to 
hold for five seconds. This is the interval of predictive 
sufficiency: seeing a green light allows the system to 
guarantee at least five more seconds of safe crossing 
time. Because the process of sensing the green light 
and then crossing the street takes no more than three 
seconds, the interval of predictive sufficiency is long 
enough to cover the sense/act gap. Therefore, CIRCA 
can plan this action and guarantee that it will only 
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be executed in appropriate situations’. 

When CIRCA continues the planning process and 
tries to choose an action for the yellow state C, it 
finds that the cross-intersection action is appli- 
cable and leads to the desired state. However, when 
the system tries to ensure that the “safe2X” precon- 
dition can be predicted to hold while the action is 
executed, it finds that a temporal transition leaving 
state C leads to the red state A, which is “unsafe2X.” 
Therefore, since the system does not know how much 
time may have passed in the yellow state C before 
the state was detected, and the subsequent state does 
not satisfy the action’s preconditions, the action is re- 
jected. In summary, CIRCA has used its explicit un- 
derstanding of predictive sufficiency to derive a com- 
mon rule of thumb used by drivers who glance at a 
traffic signal: if the light is green, go ahead and cross; 
if the light is yellow, do not start crossing, because 
the light may turn red too soon. 

An interesting feature of this approach to avoiding 
inappropriate actions is that it requires no informa- 
tion about how frequently a particular sensory ob- 
servation is being acquired — the example said noth- 
ing about how often the system checks to see if the 
light is green. If the system never even checks to 
see if the light is green, and thus never takes the 
cross-intersection action, it will never perform 
an inappropriate action. Clearly, this type of proof 
is only useful for goals that have no deadline. For 
real-time goals, that require response-time guaran- 
tees, this method is not sufficient. 

To describe CIRCA J s approach to meeting such 
real-time deadlines, we first introduce a more com- 
plex application domain. 

The Puma Domain 

The stoplight domain was used above for its intu- 
itive simplicity; CIRCA has also been applied to a 
much larger robot control problem, illustrated by the 
simulation image in Figure 3. The Puma is assigned 
the task of packing parts arriving on the conveyor 
belt into the nearby box. Once at the end of the belt, 
each part remains motionless until the next part ar- 
rives, at which time it will be pushed off the end of 
the belt (unless the robot picks it up first). If a part 
falls off the belt because the robot does not pick it 
up in time, the system is considered to have failed. 
Thus, the arriving parts impose hard deadlines on 
the robot’s responses; it must always pick up arriving 
parts before they fall off the conveyor. 

The Puma is also responsible for reacting to an 

’CIRCA currently only supports this test for precon- 
ditions that are required over the entire duration of an 
action. 



Figure 3: The Puma domain, with two hard real- 
time deadline constraints. 


emergency alert light. If the light goes on, the system 
has only a limited time to push the button next to the 
light, or the system fails. This portion of the domain 
represents a completely asynchronous interrupt with 
a hard deadline on its service time. 

Real-Time Response Guarantees 

To deal with the hard deadlines in the Puma do- 
main, the planning methods described above are not 
sufficient — they do not ensure that reactions will be 
timely, but rather that they will never be inappropri- 
ate. As we shall see, CIRCA must merge even more 
knowledge with its sensing information to guarantee 
timely responses that meet hard deadlines. 

Figure 4 illustrates a small portion of the world 
model for the Puma domain*, showing the represen- 
tation of the hard deadline on picking up arriving 
parts. Parts are known to be spaced apart on the 
conveyor by at least some minimum distance. After 
a part arrives, the conveyor belt is considered to be 
“busy” for some amount of time (corresponding to 
the minimum part spacing) before the next part may 
arrive. Thus, from state A (where CONVEYOR- 
STATUS is BUSY) there is a temporal transition 
to state B (where CONVEYOR-STATUS is FREE), 
tagged with the value mm A =10 (seconds) to indi- 
cate that state A must persist at least that long be- 
fore the transition to state B . From state B ) an event 
transition represents the fact that a part may arrive 

^The full domain model includes more state features 
and hundreds of states and transitions. 
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at any time, leading to state C. The potential fail- 
ure resulting from the part falling off the conveyor is 
represented by the temporal transition out of state C, 
also tagged with min A — 10: if the next part arrives 
while this part is still on the conveyor, failure will 
occur. 

To understand CIRCA ’s approach to making 
response-time guarantees, let us examine the plan- 
ner’s operation when it is considering state C. The 
first phase of the planning process finds applica- 
ble event and temporal transitions, and recognizes 
that there is a potential temporal transition to fail- 
ure. Since the failure is defined to be catastrophic, 
CIRCA realizes that it must preempt the tempo- 
ral transition. That is, CIRCA decides it must ex- 
ecute some action that will definitely occur before 
the earliest time the temporal transition to failure 
can occur. A simple lookahead shows that the action 
pickup-part -from- conveyor will successfully avoid 
the failure. Now the only challenge is to ensure that 
the action will happen quickly enough. To ensure that 
the transition to failure is preempted, CIRCA com- 
mits to repeatedly executing a reaction that checks 
for the conditions of state C and implements the cho- 
sen action, at least frequently enough to ensure that 
the action will be completed before failure can occur. 
That is, CIRCA decides how quickly it must poll the 
sensors to detect the imminent failure and prevent it. 

It is fairly obvious that, to guarantee that the sys- 
tem will simply detect the potential failure repre- 
sented by state C, which has a minimum possible du- 
ration (mindur(P)) of 10 seconds, CIRCA must test 
for the state at least once every 10 seconds. How- 
ever, detecting the state C is not sufficient: the system 
must be able to finish the action of picking up the part 
before it can fall off the conveyor. In the terms intro- 
duced previously, the interval of predictive sufficiency 
during which the part is known to remain on the con- 
veyor must cover the chosen action, in addition to its 
preconditions. To provide this predictive sufficiency, 
CIRCA relies on its additional knowledge about the 
frequency with which CIRCA itself will be obtaining 
sensory information. For example, if the period of the 
repeated observations is p(0) seconds, then an obser- 
vation in which the condition does hold, following an 
observation in which the condition does riot hold, in- 
dicates that the change of state must have occurred 
in the last p(0) seconds. Therefore, the condition 
must continue to hold for at least mindur(P) — p(O) 
seconds. 

Thus we have a modified interval of predictive suf- 
ficiency, based on both knowledge of the domain and 
knowledge about the ongoing performance of the re- 
active system itself. The AIS actually reasons about 


the performance of the reactive system it is design- 
ing to derive the predictive sufficiency of the observa- 
tions it plans to make. To guarantee that every real- 
time reaction will be checked and executed before its 
corresponding deadline, CIRCA must show that the 
predictive sufficiency of the observations covers the 
sense/act gap and the duration of the chosen action. 
That is, mindur(P) — p(0) > t a p — U. In our Puma 
domain example, if the pickup-part-from-conveyor 
action takes 3 seconds, we have 10 — p(0) > 3, so that 
p(0) < 7. If CIRCA can guarantee to execute the re- 
action that tests for state C and picks up the part at 
least once every 7 seconds, it can guarantee that it 
will not drop any parts off the conveyor*. 

Making this reaction frequency guarantee is the 
job of CIRCA’s Scheduler module (see Figure 1). 
The AIS uses the methods described above to de- 
rive frequency requirements for mission-critical reac- 
tions, and sends those reactions to the Scheduler. The 
Scheduler examines the capacity of the RTS to see if 
the available resources are sufficient to meet those re- 
quirements: if so, a schedule of reaction executions 
is returned to the AIS. If the RTS resources are not 
sufficient to guarantee the reaction rates specified by 
the AIS, the Scheduler will return an error message 
to the AIS, indicating that some performance tradeoff 
will be required in this over constrained domain. 

Knowledge Requirements 

As we have noted, predictive sufficiency can only 
be established by combining immediate sensor infor- 
mation with additional knowledge about the domain. 
The basic form of the required knowledge is the “min- 
imum duration” of some condition. That is, the sys- 
tem must know that some sensed state of the envi- 
ronment always persists for some minimum amount 
of time. In the stoplight domain, for example, the 
system must know the minimum duration of each sig- 
nal color. In general, this type of knowledge might be 
acquired in one of two ways. 

First, the system might have previous experience 
with the domain (or similar domains), and be able to 
extrapolate from that experience the requisite min- 
imum durations. Experienced drivers know that no 
green light lasts for less than 5 seconds. Learning and 
past experience can thus play a key role in reasoning 
about predictive sufficiency. 

Second, knowledge of minimum durations may also 
be derived from simple first principles, given precur- 
sor knowledge of the maximum rate of related (under- 
lying) processes. For example, in the Puma domain, 
the minimum duration of the (CONVEYOR-STATUS 

*At least, not from this particular part of the state 
space. 
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min A = 10 seconds 


Figure 4: A small, abstracted portion of the Puma domain model. 


BUSY) condition is determined by the maximum part 
arrival rate, which in turn is based on the conveyor 
belt speed and the spacing between parts. So if the 
system knows that parts must be at least ten inches 
apart and that the belt is moving at one inch per 
second, then the maximum part arrival rate is six 
parts per minute, and the minimum duration of the 
(CONVEYOR-STATUS BUSY) condition is ten sec- 
onds. 

Currently, CIRCA makes no effort to learn 
minimum-duration knowledge itself, and it has only 
rudimentary, domain-specific methods to derive that 
knowledge from process rates. Instead, our focus has 
been on having CIRCA use that knowledge to reason 
about predictive sufficiency, and investigating the ef- 
fects of explicitly dealing with the sense/act gap. 

Conclusion 

We have argued that all computing systems must 
make predictions about how the state of the world 
will evolve during the delay between sensing and ac- 
tion. The intuition behind the trend toward reac- 
tive systems has been that reducing this delay sim- 
plifies (but does not eliminate) prediction. In this 
paper, we have described how this intuition is really 
attempting to capture implicitly the concept of pre- 
dictive sufficiency. By explicitly representing and rea- 
soning about predictive sufficiency, we can determine 
exactly how long a gap between sensing and acting is 
allowable within a system, given its environment and 
its capabilities. 

Predictive sufficiency is a critical concept for em- 
bedded agents, because it permits a system to make 
guarantees about its behaviors. We have shown how 
CIRCA implements predictive sufficiency to guaran- 
tee that it will not execute inappropriate actions and 
that it will react to its environment frequently enough 
to meet real-time deadlines. 

Explicitly reasoning about predictive sufficiency 
also allows us to break away from the mind-set that 
decreasing the delay between sensing and acting is al- 
ways desirable. Specifically, knowing the predictive 
sufficiency of an observation may allow a system to 


avoid some sensor polling by caching sensory data. No 
sensor readings need to be taken as long as a previous 
observation’s interval of predictive sufficiency remains 
in force. We are investigating ways in which CIRCA 
can use its explicit knowledge of predictive sufficiency 
to design sensor caching schemes that maximize the 
use it gets out of each observation, reducing the fre- 
quency of costly observations without compromising 
the system’s performance guarantees. 

Our investigation of predictive sufficiency is a first 
step towards a more complete understanding of ex- 
actly when stored internal state is useful, and when 
it can lead to invalid predictions and failures. We 
hope to unify this approach with the epistemic proofs 
of Rosenschein and Kaelbling 11, 12 to establish a full 
theory of the correspondence between a system’s in- 
ternal state, its predictions, and the world. This the- 
ory would allow strong prescriptive statements about 
when and how to use stored internal state. 
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